DOCUMENT RESUME 



ED 404 363 



TM 026 433 



AUTHOR 

TITLE 



INSTITUTION 
SPONS AGENCY 

REPORT NO 
PUB DATE 
NOTE 

AVAILABLE FROM 



PUB TYPE 



Brick, J. Michael; And Others 

Adjusting for Coverage Bias Using Telephone Service 
Interruption Data* National Household Education 
Survey of 1993. Technical Report. 

Westat, Inc., Rockville, MD. 

National Center for Education Statistics (ED) , 
Washington, DC. 

ISBN-0-16-048917-2; NCES-97-336 
Dec 96 
43p . 

U.S. Government Printing Office, Superintendent of 
Documents, Mail Stop: SSOP, Washington, DC 
20402-9328. 

Statistical Data (110) — Reports - 
Evaluat ive/Feas ibi 1 i ty (142) 



EDRS PRICE 
DESCRIPTORS 

IDENTIFIERS 



MF01/PC02 Plus Postage. 

’’"Data Collection; National Surveys; Probability; 
Research Methodology; ’‘"Responses; ^Sample Size; 
Sampling; ’’"Statistical Bias; ’'"Telephone Surveys 
’""National Household Education Survey; Random Digit 
Dialing; Weighting (Statistical) 



ABSTRACT 

The National Household Education Survey (NHES) is a 
data collection system of the National Center for Education 
Statistics. The NHES is a telephone survey of the 

noninstitutionalized civilian population using households selected 
using random digit dialing methods. Approximately 60,000 households 
are screened for each administration, and people who meet 
predetermined criteria are sampled for more detailed or extended 
interviews. This report is a continuation of research on issues 
related to biases that result from the inability to survey people who 
are in households without telephones. It describes a study of an 
alternative method for adjusting telephone survey data to compensate 
for coverage bias. The method is based on the observation that 
telephone subscription varies within households over time. Weighting 
adjustments that use the data from households that have telephones 
only sometimes during the year might be an improvement over the 
current way of adjusting data. Findings indicate that coverage bias 
associated with households without telephones could be important for 
some statistics, and data collected on telephone service 
interruptions can be used to reduce this bias by using a response 
probability type of adjustment. The benefits of the bias reduction 
appear to be large enough to offset the variance increases due to 
increased variability in the weights in the study, although the 
results may differ for different size samples. Further research is 
needed, however, before the procedure can be recommended. (Contains 8 
tables, 7 figures, and 10 references.) (SLD) 
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Introduction 

The National Household Education Survey 
(NHES) is a data collection system of the 
National Center for Education Statistics (NCES), 
which has as its legislative mission the collection 
and publication of data on the condition of 
education in the Nation. The NHES is 
specifically designed to support this mission by 
providing information on those educational 
issues that are best addressed by contacting 
households rather than schools or other 
educational institutions. The NHES provides 
descriptive data on the educational activities of 
the U.S. population and offers policymakers, 
researchers, and educators a variety of statistics 
on the condition of education in the United 
States. 

The NHES is a telephone survey of the 
noninstitutionalized civilian population of the 
U.S. Households are selected for the survey 
using random digit dialing (RDD) methods, and 
data are collected using computer-assisted 
telephone interviewing (CATI) procedures. 
These procedures provide a cost effective means 
for quickly surveying households with 
telephones. Approximately 60,000 households 
are screened for each administration, and 
individuals within households who meet 
predetermined criteria are sampled for more 
detailed or extended interviews. The data are 
weighted to permit estimates of the entire 
population. The NHES survey for a given year 
typically consists of a Screener, which collects 
household composition and demographic data, 
and extended interviews on two substantive 
components addressing education-related topics. 
In order to assess data item reliability and inform 
future NHES surveys, each administration also 
includes a subsample of respondents for a 
reinterview. 

The primary purpose of the NHES is to conduct 
repeated measurements of the same phenomena 



at different points in time, although one-time 
surveys on topics of interest to the Department of 
Education are also conducted. This has been 
done by repeating topical components on a 
rotating basis to provide comparative data across 
survey years. In addition, each administration of 
the NHES has benefited from experiences with 
previous cycles, resulting in enhancements to the 
survey procedures and content. Thus, while the 
survey affords the opportunity for tracking 
phenomena across time, it is also dynamic in 
addressing new issues and including conceptual 
and methodological refinements. 

A new design feature of the NHES program 
implemented in the NHES:96 is the collection of 
demographic and educational information on 
members of all screened households, rather than 
just those households potentially eligible for a 
topical component. In addition, this expanded 
screening feature includes a brief set of questions 
on an issue of interest to education program 
administrators or policymakers. The total 
Screener sample size is sufficient to produce 
state estimates of household characteristics for 
the NHES: 96. 

Full-scale implementations of the NHES have 
been conducted in 1991, 1993, 1995, and 1996. 
Topics addressed by the NHES:91 were early 
childhood education and adult education. The 
NHES:93 collected information about school 
readiness and school safety and discipline. The 
1991 components were repeated for the 
NHES:95, addressing early childhood program 
participation and adult education. Both 
components underwent substantial redesign to 
incorporate new issues and develop new 
measurement approaches. In the NHES:96, the 
topical components are parent/family 
involvement in education and civic involvement. 
The NHES: 96 expanded screening feature 
includes a set of questions on public library use. 

In addition to its topical components, the NHES 
system has also included a number of 
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methodological investigations. These have 
resulted in technical reports and working papers 
covering diverse topics such as telephone 
undercoverage bias, proxy reporting, and 
sampling methods. This series of technical 
reports and working papers provides valuable 
information on ways of improving the NHES, 
and may be useful to survey researchers more 
generally. 

This report is a continuation of research on 
issues related to biases that result from the 
inability to survey persons who live in 
households without telephones. Two of the 
earlier NHES technical reports (Brick, Burke, 
and West 1992; Brick and West 1992) addressed 
this important subject. Another bias study 
involved adding certain questionnaire items to 
the NHES:93 to evaluate a different method of 
adjusting the estimates to reduce the bias 
associated with sampling only persons living in 
households with telephones. The method 
involves using data on interruptions of 
telephone service to adjust the weights of the 
respondents to the survey. The weights for 
households that report experiencing some 
periods of not having telephone service during 
the twelve months prior to the interview are 
increased whereas households reporting no 
breaks in telephone service receive their normal 
weights. The assumption behind this procedure 
is that households with interrupted telephone 
service are more like those without telephones 
than other telephone households. Although the 
goal of these adjustments is to reduce the bias 
due to excluding households without telephones 
at the time of the survey, a consequence of the 
adjustments is that the variances of the estimates 
increase. This analysis examines the benefits of 
the bias reduction in light of the variance 
increases and suggests situations in which the 
adjustments might be beneficial. 

The next section provides background 
information on telephone coverage bias, its 
implications for estimates from a survey such as 



the NHES, and previous research using data on 
telephone service interruptions to reduce 
coverage bias. Subsequent sections describe the 
estimates from the NHES:93 of the percentage of 
persons that experienced some interruption of 
telephone service, the procedures used to adjust 
the survey weights using these data, and the 
statistical implications of using the adjusted 
weights. The final section summarizes the 
findings and contains recommendations for use 
of this technique. 

Background 

Telephone surveys provide a relatively 
economical method of data collection compared 
with personal interviewing. However, telephone 
surveys are subject to an important source of bias 
that does not affect household surveys conducted 
with face-to-face interviewing: only 94 1 percent 
of households nationally have telephone service 
at any given time. Moreover, for the children 
surveyed for the two components of the 
NHES:93, coverage rates are lower than 94 
percent. Indeed, persons under 6 years of age 
have the lowest telephone coverage rate of all 
age groups in the U.S. (Thomberry and Massey 
1988). 

Weighting that includes poststratification based 
on demographic variables known to be associated 
with telephone coverage is effective in mitigating 
some of the consequences of coverage bias in 
telephone surveys generally, and has been shown 
to do so for many items in the NHES (Brick, 
Burke, and West 1992). 2 But even when 
effective, weighting to known demographic totals 
does not completely solve the problem of 
coverage bias. It undercompensates for some 



1 Estimate based on tabulations from the March, July, and 
November 1992 Current Population Survey. 

2 Postsurvey weighting is also used to compensate for 
nonresponse and other biases. 
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variables (Massey and Botman 1988) and 
overcompensates for others (Brick, Burke, and 
West 1992). 

This report describes a study of an alternative 
method for adjusting telephone survey data to 
compensate for coverage bias. This method is 
based on the observation that telephone 
subscription not only varies across households in 
the population, but also within households over 
time. Keeter (1995) discusses this idea in some 
depth and demonstrates that a sizable number of 
U.S. households lose and gain telephone service 
during a given year. Because of this 
phenomenon, the telephone population at a given 
time includes households that recently were in 
the nontelephone population and excludes some 
households that were recently in the telephone 
population. Thus, weighting adjustments that 
use the data from households that have 
telephones only sometimes during the year might 
be an improvement over the current practice. 

Despite considerable information on the size and 
characteristics of the nontelephone population, 
little is known about its dynamics over shorter 
time periods. Evidence from social workers, 
telephone companies, and others who deal with 
indigent households suggests that for many 
families, telephone subscription is episodic. 
They have a telephone when they can afford it; 
the telephone is turned off when times are 
harder, when the bills get too large to manage, or 
both (Federal Communications Commission 
1988). It is not known how many households 
change their telephone status and how long they 
stay in a particular status. 

Keeter (1995) examined two household panel 
surveys to obtain estimates of the dynamics of 
telephone service subscription. Those 
households that changed telephone status 
(presence of a telephone in the household) from 
one wave to the next of the survey are called 
‘transient’ households. For data collected 12 
months apart, half of the 6 percent of all 



households without a telephone at either time 
were transient. For data collected only two 
months apart, one-fourth of the 6 percent of 
households without telephones at either point in 
time were transient. Since these estimates were 
based on observations at two points in time 
rather than continuous measurement, they 
underestimate the percent of households that are 
transient. Nevertheless, these results show that a 
substantial proportion of households without a 
telephone at a specific point in time are transient. 

Another important condition that must be 
satisfied if the transient telephone households are 
to be useful in reducing coverage bias involves 
the characteristics of transient households and 
households without telephones at the time of the 
interview. If the two groups are not similar, then 
the adjustments will not be effective. Using the 
panel data and data from several Virginia 
surveys, Keeter (1995) showed that the 
characteristics of the transient households are 
more consistent with the nontelephone 
households than telephone households. 
Preliminary results on this comparison were 
presented by Keeter in a paper at the 1992 
meetings of the American Association of Public 
Opinion Research. This presentation was the 
catalyst for the inclusion of the items on the 
interruption of telephone service in the 
NHES:93. 

Estimates of Interruptions of 
Telephone Service 

In the NHES:93, 64,000 households completed 
the screening interview and nearly 30,000 
interviews were conducted within those screened 
households. Two survey components were 
included: School Readiness (SR) and School 
Safety and Discipline (SS&D). Approximately 
11,000 parents of 3- to 7-year-olds completed 
interviews on SR topics, including 
developmental characteristics of preschoolers. 
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school ■ adjustment and teacher feedback to 
parents for kindergartners and primary students, 
home activities with family members, and health 
status. About 12,700 parents of children in 
grades 3 through 12 and about 6,500 youth in 
grades 6 through 12 were interviewed for the 
SS&D component. The topics for this 
component included the school learning 
environment, safety at school, and availability 
and use of tobacco, alcohol, and other drugs at 
school. For both components, characteristics of 
the family and household background 
information were collected. 

The SR component included the 20 million 
children between the ages of 3 and 7 years as of 
December 31, 1992, and all other children 
through age 9 who were enrolled in kindergarten, 
first, or second grade. The SS&D component 
included the 35 million students in grades 3 
through 12. The estimates of the population 
were derived from the October 1992 Current 
Population Survey (CPS). 

For all households that completed an interview, 
one parent was asked if the household had 
experienced an interruption in telephone service 
in the last 12 months, where an interruption is 
any 24 hour period without telephone service. If 
the respondent said yes, he or she was asked how 
many days, weeks, or months the household was 
without service. This question was asked only of 
one parent in the household, even if there were 
multiple interviews in the household. (See 
Exhibit 1 for the interview questions.) The 
responses 3 to these items are the basis for the 
study of the effects of adjustments for telephone 
coverage discussed in the rest of this report. 

Since the responses to these questions in the 
NHES:93 were only obtained for those 
households that completed either an SR or SS&D 



3 The imputed responses were used for records with 
missing values. Only 123 of the 10,888 SR and 71 of the 
12, 680 SS&D values were imputed. 



interview, this has implications for the analysis 
of the results. The data presented below pertain 
only to persons in certain households: those in 
which there was at least one child from preschool 
age (at least 3 years old) to the end of high 
school. Since the two eligible populations in the 
NHES:93 are not overlapping, the estimates are 
presented separately for the SR and SS&D 
children. In addition, the estimates are of 
children rather them households. This is an 
important distinction. Since the estimates from 
the NHES:93 generally refer to children rather 
than households, the impact of the coverage 
adjustments should be measured at the person 
level rather them the household level. 

The estimated percentage of SR children in 
households that had a telephone interruption of 1 
day or more was 12 percent of all children in 
telephone households at the time of interviews, 
while it was only 9 percent for the SS&D 
children. This estimated difference in the 
percentage with telephone service interruptions 
between the two populations is consistent with 
estimates that find lower telephone penetration 
for younger children. Thomberry and Massey 
(1988) reported that 12.3 percent of children 
under 6 years were in nontelephone households 
while only 8.5 percent of those 6 to 16 years 
were in nontelephone households. 

Figure 1 shows the estimated percentage of 
persons in each population who had a service 
interruption by the length of the interruption. 
The vertical lines in the figure are 95 percent 
confidence intervals on the estimated 
percentages. Intervals constructed using these 
methods include the population value in 95 
percent of all possible samples that could be 
selected. Both populations exhibit roughly the 
same pattern in the estimates by length of 
service. A substantial proportion of those with 
interruptions in telephone service experience 
only short interruptions of less than 1 week. 



Exhibit 1 

Telephone Interrupt Items 



1. During the past 12 months, has your household ever been without telephone service for more than 24 
hours? 



YES 

NO 

REFUSED 

DON’T KNOW 



1 (GO TO 2) 

2 
-7 
-8 



2. What was the total amount of time your household was without telephone service in the past 12 
months? 



NUMBER CD 

DAYS.. 1 

WEEKS -2 

MONTHS -3 



Figure 1.-- Estimated 95 percent confidence intervals of the percentage of persons with interrupted 
telephone service during the previous 12 months, by length of interruption 
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SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Survey, spring 1993. 
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Characteristics of Those With Service 
Interruptions 

The estimated percentage of children in 
households with any interruption in service is 
given in table 1 for the SR population and table 2 
for the SS&D population. Characteristics 
collected for both populations are the first items 
shown in the tables. 

The purpose of examining the telephone 
interruption estimates by the characteristics of 
the children is to evaluate the potential of using 
the data to adjust for nontelephone coverage bias. 
If the percentages of persons in households with 
telephone service interruptions are nearly the 
same for all persons across the characteristics, 
then little could be expected from using the items 
to reduce coverage bias. 

The percentage distributions of persons with 
some interruption for the nine common items are 
relatively consistent for the SR and the SS&D 
populations. All of the items, except Census 
region and community mobility in the ZIP Code 
area, exhibit variation 4 in the percentage with 
interruptions. The characteristics associated with 
lower economic status have the highest 
percentage with interruptions in all of these 
common items: the percentage of black and 
Hispanic children in households with 
interruptions is higher than for white children; 
the percentage of households with interruptions 
for those renting is higher than for those who 
own; the percentage with household incomes less 
than $20,000 is higher than for those with larger 
incomes; the percentage for those from 
households with lower parental education levels 
(only completed high school or less) is higher 
than for those from households with higher 
parental education levels (college graduate or 
more); and the percentage of those living in ZIP 



4 The statements in this report were tested at the 5 percent 
significance level. Bonferroni adjustments were made to 
compensate for multiple levels of the response variables. 



Code areas with median household incomes of 
$15,000 or less is higher than for those in areas 
with household incomes of $25,000 or more. 

The remaining items in table 1 were designed to 
address specific substantive issues associated 
with school readiness. For most of the items, the 
percentages of children in households with 
telephone interruptions are not statistically 
significant for different levels of the variables. 
The most striking differences are for the 
estimates of those respondents participating in 
Women, Infant, and Children (WIC) program 
and whose children participate in the free meal or 
lunch program at school. The higher estimates of 
the percentage with telephone interruptions for 
children participating in these programs designed 
for the economically disadvantaged is consistent 
with the estimates for the other economically 
related items. For other substantive items, the 
differences in the percentage of persons with 
some interruption in telephone service are 
statistically significant, but not large enough to 
be of great practical importance. For example, 
the difference in the percentage of children in 
kindergarten or primary school who attended a 
center-based program prior to school is 
statistically significant, but the estimates differ 
by less than 3 percent. A difference of this size 
may not be important for adjusting for coverage 
bias. 

The estimates for the SS&D population in table 2 
are similar. The differences across response 
categories for most of the items are either not 
statistically significant or so small that they have 
little practical importance. For example, the 
difference between children in public and private 
school is less than 2 percent and not likely to be 
important for adjustment purposes. 

The estimates in tables 1 and 2 support the 
hypothesis that the chance of having telephone 
service interruptions is related to the economic 
situation in the household. Since race/ethnicity 
and economic status are highly correlated, a 
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relevant question is whether the differences in 
the percentage of persons with telephone 
interruptions would be significant across 
categories of race/ethnicity after controlling for 
other variables related to economic status, such 
as household income or parental education since 
poststratification by income or education is 
frequently used in RDD surveys. To examine 
this, the percentage of the SR population with an 
interruption in telephone service in the last year 
by race/ethnicity and two categories of parental 
education was estimated and is shown in 
Figure 2. The figure shows 95 percent 
confidence intervals along with each of the point 
estimates. If education level accounted for all of 
the variability in the percentage with 
interruptions, then all three low education 
estimates should be equal and all three of high 
education estimates should be equal within 
sampling error. Clearly, this is not the situation; 
the estimates for blacks and Hispanics at the high 
parental education level are greater than the 
estimates for the nonblack, non-Hispanics. 
Race/ethnicity is an important correlate even 
after controlling for parents’ education level. 
Tabulations controlling for household income 
instead of parental education were also prepared, 
and the estimates are also statistically significant. 
The same results also hold for the SS&D 
population controlling for either parental 
education or household income. 

These findings indicate that interruptions in 
telephone service as estimated from the 
NHES:93 do vary by economic and demographic 
characteristics that have been identified as 
important correlates of telephone coverage. This 
condition is necessary for the telephone 
interruption data to be useful in adjusting for 
telephone coverage bias. The lack of important 
differences for many of the substantive items, 



especially in the SS&D population, suggests the 
value of the adjustment may be less important for 
estimates of the substantive items. 

Weighting Adjustments 

In most sample surveys, the data collected from 
respondents are processed to make the estimates 
more representative of the population surveyed. 
A typical operation is to attach a survey weight 
to each observation and use these weights in the 
preparation of estimates. The weights are often 
the product of several steps. A base weight that 
is the reciprocal of the probability of including 
the respondent in the sample is first attached to 
each record. The base weight is then adjusted to 
account for nonresponse and noncoverage and to 
reduce the variability in the estimates by using 
auxiliary data. 

Kalton and Kasprzyk (1986) discuss adjustments 
to the base weights, classifying the adjustments 
into four categories: population weighting 

adjustments, sample weighting adjustments, 
raking ratio adjustments, and response 

probability adjustments. In the NHES:93, 
sample weighting adjustments and raking ratio 
adjustments were used. Sample weighting 
adjustments were used to account for differential 
nonresponse from sampled persons. Raking ratio 
adjustments were then used to make the specified 
marginal distributions of the sample correspond 
to totals from the October 1992 CPS. One of the 
most important benefits of the type of raking 
ratio adjustment used in the NHES:93 is that it 
reduces the bias associated with the 

undercoverage of persons living in households 
without telephones because the CPS covers 
persons in both telephone and nontelephone 
households. 
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Table 1.-- Estimated percentage of persons in the School Readiness population 1 with interruptions in 
telephone service in last 12 months, by selected characteristics 



Characteristic 


Estimate 


Standard error 




12.0 


0.4 


Race/ethnicity 


9.3 


0.5 


White, non-Hispanic 


Black, non-Hispanic 


19.8 


1.5 


Hispanic 


17.2’ 


1.5 


Other 


11.7 


2.6 


Tenure 






Own/other. 


7.9 


0.5 


Rpnt 


18.4 


1.0 


Household income 






$10,000 or less 


22.8 


1.3 


$10,001 to $20, 000 


19.9 


1.4 


$20,001 to $30,000 


9.3 


0.8 


More than $30,000 


5.5 


0.5 


Parental educational level 






Less than high school graduate 


18.4 


1.8 


High school graduate or equivalent 


15.4 


0.8 


Vocational/some college 


11.8 


0.7 


College graduate • 


5.5 


0.8 


Graduate school 


5.2 


0.7 


Mother's employment status 


17.6 


3.5 


No mother in household 


Employed 35 hours/week or more 


10.1 


0.7 


Employed less than 35 hours/week 


9.6 


0.9 


Seeking employment 


20.7 


2.2 


Not in labor force 


13.1 


0.7 


Father's employment status 


18.2 


1.1 


No father in household 


Employed 35 hours/week or more 


8.7 


0.4 


Employed less than 35 hours/week 


15.4 


2.6 


Seeking employment 


19.6 


3.4 


Notin labor force 


14.8 


2.6 


Census region 




1.2 


Northeast 


9.5 


South 


13.6 


0.7 


Midwest 


11.1 


1.0 


West 


12.5 


0.9 


Median household income in ZIP Code 






$15,000 or less 


18.3 


3.0 


$15,000 to $25,000 


15.8 


0.9 


More than $25,000 


9.9 


0.5 


Mobility in ZIP Code 








13.0 


1.9 


Medium/high 


13.1 


1.3 


Medium 


12.2 


0.9 


Medium/low 


11.0 


0.8 


I jOW 


11.6 


1.4 


Time since doctor visit for routine care 




0.5 


Less than 1 year 


11.8 


Over 1 year 


13.1 


1.0 
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Table 1 Estimated percentage of persons in the School Readiness population 1 with interruptions in 
telephone service in last 12 months, by selected characteristics (continued) 



Characteristic 


Estimate 


Standard error 


Birth weight 




1.6 


5.5 pounds or less 


12.0 


Greater than 5.5 pounds 


12.0 


0.4 


Child attending center-based program 2 






Yes 


9.3 


0.7 




13.7 


1.1 


Child ever attended center-based program 2 






Yes 


10.5 


0.7 




13.0 


1.2 


Child ever attended center-based program prior to school 3 






Yes 


11.7 


0.6 


No 


14.4 


1.2 


Women, Infant, and Children program participant* 






Yes 


18.2 


1.3 




8.0 


0.6 


Free meal at school or center* 






Yes 


21.1 


1.2 


No 


7.6 


0.5 


Repeated kindergarten 4 






Yes 


15.7 


3.5 


No 


11.7 


0.6 


Family member read to child in last week? 






Not in last week 


21.9 


5.4 


Once or twice 


11.9 


0.8 


Three or more times 


11.5 


0.9 


Family member taught child letters or words in last week* 






Not in last week 


12.7 


1.8 


Once or twice 


10.5 


1.0 


Three or more times 


12.6 


0.6 


Family member taught child songs or music in last week? 






Not in last week 


12.3 


1.0 


Once or twice 


11.1 


0.9 


Three or more times 


12.4 


1.0 


Family member did arts or crafts with child in last week? 






Not in last week 


14.9 


1.0 


Once or twice 


10.9 


0.9 


Three or more times 


10.4 


0.9 


Family member visited library with child in last month 5 






Yes 


10.2 


0.8 




13.2 


0.7 


Family member visited zoo with child in last month 5 






Yes 


10.6 


1.3 




12.3 


0.6 



‘The SR population is approximately 20 million children from 3 to 7 years old. 

2 Estimate restricted to preschoolers. 

■fetimate applies to all children except preschoolers. 

'^Estimate restricted to children in primary school. 

5 Estimate applies to all children except those in primary school. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Survey, spring 1993. 
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Table 2.- Estimated percentage of persons in the School Safety and Discipline population 1 with 
interruptions in telephone service in last 12 months, by selected characteristics 



Characteristic 


Estimate 


Standard error 




9.2 


0.3 


Race/ethnicity 






White, non-Hispanic 


7.2 


0.3 


Black, non-Hispanic 


14.7 


1.1 


Hispanic 


14.1 


1.1 


Other 


9.3 


1.5 


Tenure 






Own/other 


6.6 


0.3 


Rent 


15.3 


0.8 


Household income 






$10,000 or less 


19.0 


1.3 


$10,001 to $20,000 


15.7 


1.1 


$20,001 to $30,000 


7.9 


0.6 


More than $30,000 


5.0 


0.3 


Parental educational level 






Less than high school graduate 


17.4 


1.6 


High school graduate or equivalent 


11.0 


0.8 


Vocational/some college 


8.6 


0.5 


College graduate 


5.3 


0.8 


Graduate school 


4.5 


0.6 


Mother's employment status 






No mother in household 


12.8 


1 9 


Employed 35 hours/week or more 


8.4 


1 .<7 

0.5 


Employed less than 35 hours/week 


7.8 


0.6 


Seeking employment 


15.1 


1.6 


Not in labor force 


10.3 


0.7 


Father's employment status 






No father in household 


12.9 


0.9 


Employed 35 hours/week or more 


6.8 


0.3 


Employed less than 35 hours/week 


14.6 


2.6 


Seeking employment 


17.3 


2.5 


Not in labor force 


13.8 


1.6 


Census region 






Northeast 


9.0 


0.8 


South 


10.8 


0.6 


Midwest 


7.3 


0.7 


West 


9.2 


0.8 


Median household income in ZIP Code 






$15,000 or less 


15.4 


2.1 


$15,000 to $25, 000 


11.6 


0.8 


More than $25,000 


7.7 


0.3 


Mobility in ZIP Code 


High 


7.6 


1.7 


Medium/high 


9.7 


O Q 


Medium 


9.1 


U.7 

0.6 


Medium/low 


9.2 


0.5 


Low 


9.6 


1.0 


School control 






Public 


9.4 


0.4 


Private ; 


7.5 


1.1 


Visitors required to sign in at school 






Yes 


9.4 


0.4 


No 


8.6 


0.7 
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Table 2.- Estimated percentage of persons in the School Safety and Discipline population 1 with 
interruptions in telephone service in last 12 months, by selected characteristics (continued) 



Characteristic 


Estimate 


Standard error 


Students in fighting gangs at school 2 






Yes 


9.5 


0.8 


No 


8.8 


0.5 


Ease of obtaining marijuana at school 2 






Very or fairly easy 


9.7 


0.6 


Hard 


8.0 


0.8 


Nearly impossible 


9.0 


0.7 


Had drug or alcohol ed program this year 






Yes 


8.5 


0.4 


No 


10.9 


0.7 


Fear of incident of crime at school 






None 


8.7 


0.4 


Fear of theft or robbery 3 


9.4 


1.0 


Fear of bullying or assault 3 


11.5 


1.2 


Fear of two or more types of incidents 3 


10.6 


0.9 


Knowledge of crime at school 






None 


8.9 


0.5 


Knowledge of theft or robbery 3 


7.5 


0.8 


Knowledge of bullying or assault 3 


10.8 


0.9 


Knowledge of two or more types of incidents 3 


9.6 


0.5 


Victimization by crime 






Not victimized 


8.6 


0.4 


Victim of theft or robbery 3 


10.8 


1.0 


Victim of bullying or assault 3 


11.2 


1.2 


Victim of two or more types of incidents 3 


10.6 


1.2 


Witnessed crime at school 






None 


8.8 


0.5 


Witnessed robbery 


8.5 


4.1 


Witnessed bullying or assault 4 


10.1 


0.5 


Witnessed two or more types of incidents 


10.1 


0.9 



] The School Safety and Discipline population is approximately 35 million students in grades 3 through 12. 

2 Only asked for students in grades 6 through 12. 

3 For the fear of incident, knowledge of crime, and victimized by crime variables, the second response category is used if either theft or robbery was 
reported but not both, the third response category is used if either bullying or assault was reported but not both. 

4 This response category is used if either bullying or assault was reported, but not both. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Survey, spring 1993. 
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Figure 2.-- Estimated 95 percent confidence intervals of the percentage of School Readiness 
population with service interruption 



c 




nonHispanic nonHispanic 

Race/ethnicity 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Survey, spring 1 993. 



The data on telephone service interruptions can 
be used to make a response probability 
adjustment. Response probability adjustments 
are constructed by assuming that each sampled 
unit has a probability of responding to the 
survey, estimating that probability, and then 
using the inverse of the response probability as a 
weighting adjustment. The Politz and Simmons 
(1949) method is probably the best known 
application of the response probability 
adjustment procedure. 

To apply this type of adjustment with the 
telephone service interruption data, assume that 
living in a telephone household is a dynamic 
phenomenon, and that a probability distribution 
can be associated with this status. Conceptually, 
a survey is conducted by sampling from this 
distribution and observing only those members 
that live in telephone households at the time of 
the survey. The probability of living in a 
telephone household must then be estimated for 
each respondent. The inverse of the estimated 
probability is the coverage adjustment. This 
model assumes that each person can be assigned 
a probability of being in a household with a 



telephone and that the probability is between 
zero and one (but not equal to zero). 

For this analysis, the data on whether or not a 
household had an interruption in telephone 
service and the length of that interruption are the 
basis for an adjustment, using methods suggested 
by Keeter (1995). Persons are divided into two 
categories: those in households with 

interruptions in service and those in households 
without interruptions in service. The probability 
is assumed to be one for persons in households 
without interruptions and their weights are not 
adjusted. The weights of persons in households 
with at least some interruptions in the last 12 
months are adjusted to account for other 
households that have a probability of being 
covered of less than one. The adjustments may 
vary depending on the length of time they lived 
in nontelephone households and on other 
characteristics of the household. The purpose of 
having different adjustments is to account for the 
fact that some persons are more likely to live in 
nontelephone households than others. 

Although the weighting adjustments may reduce 
the undercoverage bias, introducing adjustments 



O 

ERIC 



- 12 - 



also typically increases the variances of the 
estimates. Kish (1992) discusses the reasons for 
unequal weights as well as the consequences 
from using them in a variety of situations. He 
advocates a common statistical approach of 
balancing the effect of the adjustments in 
reducing the bias of estimates against the 
increases in the variances of the estimates. If the 
weights reduce the bias of the estimates 
significantly, then it may be worthwhile 
accepting the variance increases. On the other 
hand, small reductions in bias associated with 
large variance increases are not recommended. 

In the remainder of this section, the specific 
weighting adjustment procedures examined using 
the telephone service interruption data are 
described. The methods for creating the 
adjustments and applying them to the NHES:93 
are presented in some detail. The statistical 
properties of the weights developed under four 
alternative adjustment scenarios are presented. 
The alternative weights are applied to the 
NHES:93 data and the estimated decrease in the 
bias of the estimates is compared with the 
increase in the variance of the estimates due to 
the unequal weighting. 

Adjustment Schemes 

The first step was to decide how to classify the 
length of interruption in telephone service. 
Various lengths of interruptions were examined 
to determine cut-offs which appeared to 
distinguish between temporary interruptions, not 
due to economic causes and others. It was 
decided to use two categories for forming 
adjustment cells: 1 week or more and 1 month or 
more. When tables like tables 1 and 2 were 
created using the 1 week or more and 1 month or 
more criterion rather than any interruption, the 
estimates for the transients were still highly 
related to the economic and demographic 
variables identified in tables 1 and 2. A category 



for interruptions of less than 1 week was not 
used for adjustment because short-term 
interruptions may have been caused by factors, 
such as temporary weather-related outages, that 
are different from the longer term interruptions. 

Within each of the length-of-service interruption 
categories, the children were classified into 
adjustment cells based on either parental 
education or tenure (home ownership). 
Race/ethnicity was used to form cells within the 
parental education and tenure categories. These 
cells were chosen because the percentage of 
persons with interruptions varied by these 
characteristics and the corresponding data were 
also available from the CPS. Four adjustment 
schemes were defined using these items: 

• Scheme A1 — children in households that had 
a telephone service interruption of 1 week or 
more within categories defined by parental 
education (less than high school, high school 
diploma, college diploma or above) and 
race/ethnicity (Hispanic, black/non-Hispanic, 
white and other/non-Hispanic); 

• Scheme A2 — children in households that had 
a telephone service interruption of 1 month 
or more within categories defined by parental 
education and race/ethnicity; 

• Scheme B1 — children in households that had 
a telephone service interruption of 1 week or 
more within categories defined by tenure 
(own/other, rent) and race/ethnicity; and 

• Scheme B2 — children in households that had 
a telephone service interruption of 1 month 
or more within categories defined by tenure 
and race/ethnicity. 

In these schemes, the children classified as living 
in households with interruptions of one month or 
more (A2 and B2) are a subset of those classified 
as having interruptions of one week or more. In 
other words, if the weight for a child was 
adjusted under scheme A2 or B2 it was also 
adjusted under scheme A1 or B 1 . 




- 13 - 



The adjustment factors for these schemes could 
not be obtained directly from the NHES:93 data 
because no data were collected from households 
without telephones. Instead, the adjustments 
were developed from CPS data and then applied 
to the NHES:93 weights, following the idea 
suggested by Keeter. 

To explain the adjustment of the weights under 
the response probability model assumptions, 
consider partitioning the universe of persons at 
the time of the interview into four components: tj 
is the number of persons in telephone households 
with no telephone interruptions in the past year; 
t 2 is the number of persons in telephone 
households with some telephone interruptions in 
the past year; r 3 is the number of persons in 
nontelephone households with no telephone 
interruptions in the past year; i.e., persons who 
lived in nontelephone households throughout the 
entire year and r 4 is the number of persons in 
nontelephone households with some telephone 
interruptions in the past year. As noted above, 
the response probability model assumes r 3 = 0, 
i.e., no persons live in nontelephone households 
for the entire year. This assumption is clearly 
not true, but there are no sources to estimate the 
size of r 3 . However, under the response 
probability model assumed all r 3 persons are 
included in the r 4 population. 

Using the March 1992 CPS it is possible to 
estimate t\+t 2 (but not the separate quantities) 

and r 4 ; designate these estimates as t\ + t 2 and 
f 4 , respectively. Notice that f 4 includes persons 
currently living in households without 
telephones, regardless of whether they had an 
interruption in service in the last year. Thus, it 
includes the r 3 population. From the NHES:93, 

t] and t 2 can be estimated separately; call these 
* * 

estimates t\ and t2 , respectively. The bias in 
the NHES:93 estimates arises because they do 
not include persons in nontelephone households 
(f 4 ). The goal is to reduce this bias by adjusting 

the NHES:93 weights of those persons living in 



telephone households with some telephone 
interruption. 



A weight adjustment of 



A = l+ % 



would 



result in unbiased estimates of totals under the 
response probability model, since this model 
assumes r 4 and t 2 are members of the same 

population but in different telephone status at the 
time of the interview. However, this adjustment 
involves quantities that are unknown and must be 
estimated. Since t 2 can only be estimated 
separately from the NHES:93 and r 4 can only be 
estimated from the CPS, the adjustment is 
expressed in ratios to reduce the bias due to 
estimating the quantities from different surveys. 
The revised weight is 



1 + S: 



h +f 2 
* 

h 

■ * * 
+t 2 



( 1 ) 



where w ( - is the NHES:93 weight adjusted for 

nonresponse of sampled persons but not yet 
raked to October 1992 CPS totals, 5, = 1 if the 
person lives in a household that had an 
interruption of telephone service in the last year 
and is zero otherwise. The quantity in 
parenthesis in (1) is the weight adjustment. 



Revised weights were computed separately for 
the SR and SS&D components, since these were 
handled as separate surveys. Rather than the 
overall adjustment as given in (1), the weight 
adjustments were computed within the cells 
defined for each of the four weighting schemes 
(Al, A2, Bl, and B2). Table 3 shows the 
resulting adjustment factors for the SR and 
SS&D components. The adjustments in the first 
column are those for schemes Al and Bl. The 
second column contains the adjustment factors 
for schemes A2 and B2. The adjustment factors 
for the schemes based on the 1 month or more 
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interruptions are greater than those based on the 
1 week or more because the denominator of the 
ratio is smaller for this classification. 

The last weighting step rakes the four alternative 
weights to the same October 1992 CPS totals 
used in raking the standard NHES:93 person- 
level weights. The result of this process is the 
standard NHES:93 weight and four alternative 
weights based on different adjustment schemes. 
All five of the weights conform to the same 
marginal totals. The only difference in the 
weights is the adjustment for the telephone 
service interruption. 

Effect of Adjustments on Variance 

As discussed before, the adjustment of the 
weights to reduce the bias increases the 
variability of the weights and the variance of the 
estimates. Kish (1992) gives an approximate 
expression for this increase in variance due to 
having weights that are not equal, the variance 
inflation factor ( VIF ). The VIF is a reasonable 
approximation if the population element 
variances of the persons sampled at different 
rates are roughly equal. The VIF can be written 
as 

VI F= 1+ CV 2 (weights) (2) 

where CV is the coefficient of variation of the 
weights. 

Table 4 shows the VIF for the standard NHES.93 
weights for each component. The SS&D 



component is broken down by the grade of the 
student, because youth were selected at different 
rates for these grade levels. Only one VIF is 
presented for the SR component because all 
children were sampled at the same sampling rate. 
The VIF for each of the components is about 1 .4, 
indicating the variance is inflated by about 40 
percent due to the variability in the standard 
weights. The VIF for the combined SS&D file is 
somewhat larger (1.5) because it includes youth 
sampled at different rates. The VI Fs for many 
subdomains of children should be well 
approximated by the VIF for the full component. 

The last four columns in table 4 are the ratios of 
the VIF for the four alternative weights to the 
VIF for the standard weight. These ratios show 
how much greater the variances of estimates 
produced using the alternative weights are 
expected to be as compared to the variances of 
the standard NHES:93 weights. 

Overall, the increase in variance due to the 
telephone interruption coverage adjustment are 
from 9 to 13 percent for schemes A1 and B1 in 
the SS&D component but up to 20 percent for 
the SR component. The ratios are larger for the 
schemes A2 and B2, ranging from 24 to 35 
percent, with the largest ratio for Scheme A2 for 
the SR component. The larger ratios (hence 
VIFs ) for the schemes based on interruptions of 1 
month or more are a consequence of the larger 
and more variable factors shown in the second 
column of table 3. The ratios for the SR 
population are higher than the SS&D ratios. 
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Table 3. -Weighting cell adjustments factors, based on length of interruption of telephone service 

Length of service interruption 



Factor 


1 week or more 


1 month or more 


School Readiness 






Cells defined by parental education and race/ethnicity (scheme A) 






Less than high school diploma; Hispanic 


5.75 


16.35 


Less than high school diploma; black, non-Hispanic 


5.10 


6.72 


Less than high school diploma; white and other, non-Hispanic 


4.98 


5.37 


High school diploma; Hispanic 


2.31 


2.76 


High school diploma; black, non-Hispanic 


2.65 


3.73 


High school diploma; white and other, non-Hispanic 


2.16 


2.79 


Bachelor's degree or higher; Hispanic 


1.34 


2.33 


Bachelor’s degree or higher; black, non-Hispanic 


1.77 


2.64 


Bachelor's degree or higher; white and other, non-Hispanic 


1.58 


2.09 


Cells defined by tenure and race/ethnicity (scheme B) . 






Rent; Hispanic 


3.74 


5.15 


Rent; black, non-Hispanic 


3.23 


4.54 


Rent; white and other, non-Hispanic 


2.43 


2.96 


Own/other; Hispanic 


2.00 


3.06 


Own/other; black, non-Hispanic 


2.53 


3.46 


Own/other; white and other, non-Hispanic 


2.26 


3.45 


School Safety and Discipline 






Cells defined by parental education and race/ethnicity (scheme A) 






Less than high school diploma; Hispanic 


4.89 


8.52 


Less than high school diploma; black, non-Hispanic 


4.26 


5.9.5 


Less than high school diploma; white and other, non-Hispanic 


3.81 


4.86 


High school diploma; Hispanic 


2.67 


4.51 


High school diploma; black, non-Hispanic 


3.06 


4.71 


High school diploma; white and other, non-Hispanic 


2.18 


3.09 


Bachelor's degree or higher; Hispanic 


1.96 


8.22 


Bachelor’s degree or higher; black, non-Hispanic 


1.35 


8.83 


Bachelor's degree or higher; white and other, non-Hispanic 


1.91 


3.48 


Cells defined by tenure and race/ethnicity (scheme B) 






Rent; Hispanic 


3.58 


6.08 


Rent; black, non-Hispanic 


3.38 


4.95 


Rent; white and other, non-Hispanic 


2.99 


4.00 


Own/other; Hispanic 


2.81 


5.66 


Own/other; black, non-Hispanic 


2.90 


6.11 


Own/other; white and other, non-Hispanic 


2.03 


3.10 



SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Survey, spring 1993. 
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Table 4.~Ratios of variance inflation factor due to coverage adjustment 







VIF* 

standard 


Ratio of scheme’s VIF to standard weight’s VIF 


Component 


Sample size 


weight 


Scheme A 1 


Scheme A2 


Scheme B1 


Scheme B2 


School Readiness 

School Safety and Discipline 


10,888 


1.36 


1.20 


1.35 


1.16 


1.26 


3rd through 5th graders 


2,563 


1.37 


1.12 


1.25 


1.13 


1.26 


6th through 12th graders 


10,117 


1.39 


1.13 


1.27 


1.09 


1.24 


3rd through 12th graders 


12,680 


1.49 


1.12 


1.26 


1.11 


1.25 



*' VIF is the standard inflation factor. It is the coefficient of variation of the weights squared plus one. 



SOURCE; U.S. Department of Education, National Center for Education Statistics, National Household Education Survey, spring 1993. 



Effect of Adjustments on Mean Square 
Error 

In this section, the adjusted weights are applied 
to estimate the characteristics from the SR and 
SS&D components. Since four adjustments were 
constructed, five different estimates are 
computed: one from the standard NHES.93 
weights and one for each of the four adjusted 
weights. The only difference in the methods 
used to compute the estimates for the five 
weights is the coverage adjustment. All five sets 
of weights include the final raking adjustment of 
the estimates to the totals from the CPS. 
Therefore, even the standard estimates are 
adjusted to account for undercoverage, but they 
do not have the telephone interruption 
adjustment. 

The discussion of the alternative weighting 
schemes begins by considering the reduction in 
coverage bias, the difference between the 
estimate and the value that would have been 
obtained if households without telephones could 
have been surveyed. The bias corrections are 
then compared to the increase in variance 
associated with the adjustment procedures. 



Coverage Bias Reduction 

If estimates of the same characteristics as those 
produced from the NHES:93 were available from 
an independent source and these benchmark 
estimates were free of telephone coverage bias, 
then it would be possible to compare the five 
estimates to the benchmark. The comparisons 
could be used to evaluate the bias in the standard 
NHES-.93 estimates and the bias remaining in the 
other estimates after the coverage adjustments. 
However, benchmark estimates comparable to 
the estimates from the two components of the 
NHES:93 do not exist. Consequently, other 
methods are needed to assess the bias-reducing 
potential of the coverage adjustments. 

Due to of the lack of a benchmark, some model 
assumptions are required to assess the 
effectiveness of the adjustments. For this 
evaluation it is assumed that the adjustment 
procedures eliminate the coverage bias. As a 
result of this assumption, the difference between 
the standard estimate and the adjusted estimate is 
an unbiased estimate of the coverage bias 
resulting from using the procedures. In practice, 
the coverage bias is not completely eliminated by 
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any of the adjustment procedures. Even if the 
model were correct, the bias reductions estimated 
from the data would still be subject to sampling 
error. Despite the problems with this 
assumption, it is necessary to obtain some idea of 
the effectiveness of the adjustment. If the 
adjustment eliminates the bias, the mean square 
errors of the adjusted estimates are equal to the 
variances of the estimates, with no contribution 
from coverage bias. Therefore, the model 
assumption is favorable to the adjustment 
process, positing the adjusted estimates to be 
unbiased (any difference between the standard 
and adjusted estimates are attributed to bias). 
The impact of this assumption is discussed 
critically after evidence of the effectiveness of 
the method is presented. 

The estimate from each scheme can be compared 
to the standard NHES:93 estimate, and the 
difference between the standard estimate and the 
adjusted estimate is an estimate of the reduction 
in the coverage i bias. With four adjusted 
estimates, four 'different estimates of bias 
reduction are possible. The estimated reduction 
in bias is 

b a =Ps~Pa . (3) 

where b a is the estimated bias reduction using 
adjustment scheme a (a = Al, A2, Bl, or B2), 
p s is the estimate of the proportion using the 
standard estimate, and p a is the estimated 
proportion using adjustment scheme a. 

The estimated reductions in bias under each 
adjustment weighting scheme are given in table 5 
for the SR characteristics and table 6 for the 
SS&D characteristics. It is important to 
understand that the estimates are of the amount 
of bias reduction in the standard estimate, 
assuming each adjustment scheme reduces the 
coverage bias. For example, the estimated bias 
reduction in the standard estimate of the 
percentage of preschool children attending a 
center-based program is 0.9 percent if scheme Al 
is the assumed benchmark, and 0.3 if scheme A2 



is used instead. The standard NHES:93 estimate 
of this percentage is 52.6 percent, so the scheme 
Al weight results in an estimate of 51.7 percent 
and the scheme A2 estimate is 52.3 percent. The 
bias reduction estimates in tables 5 and 6 are 
shown for all the items in the earlier tables, 
except for those items that were used in raking. 5 
Since the raking procedure forces the estimates 
to the given marginal totals for these items, the 
adjusted estimates and the standard estimate are 
all equal for these characteristics. 

The bias reduction estimates for most of the 
items in tables 5 and 6 are less than 1 percent and 
consistent across the schemes. Before 
summarizing the estimates, it is important to 
realize that the total number of children is 
constant for all the estimates due to the raking of 
the estimates to the CPS totals. The estimated 
reductions in bias across different response 
categories of an item, therefore, must sum to zero 
(positive bias reductions in response categories 
must be balanced by a negative estimates for 
other categories). As a result, the estimate and 
the bias reduction for the last category of a 
variable can be deduced from the estimates from 
the other levels. 

The fixed total number of children across 
response categories has two consequences. It 
creates a negative correlation in the estimated 
reduction in bias across response categories 
(resulting from the zero sum nature of the total) 
and gives a false impression of the number of 
independent pieces of information in the tabled 
values. For example, for a dichotomous variable, 
the bias estimates are perfectly negatively 
correlated (the estimate in one category is the 
negative of the estimate for the other category). 
Thus, there is only one independent estimate for 
a dichotomous item. 



5 Raking was done to marginal totals by age, grade, tenure 
(own, rent), Census region, race/ethnicity, and household 
income. 
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The approach taken to address this problem in 
summarizing the estimates is to delete the 
estimate for one of the response categories for 
each item. For example, in the SS&D 
component estimates in table 6, rather than 
include both the estimate of the bias for the 
percentage of students in public school (-0.1 in 
scheme Al) and private school (0.1 in scheme 
Al), only the public school estimate is retained 
for the summary. The "no" response category for 
all items with "yes" and "no" response categories 
is deleted. For other types of variables the 
response category with the smallest estimate is 
deleted. Of the 19 items in table 5 with estimates 
for 53 response levels, only 34 (53-19) are 
retained for the summaries below; of the 14 
items with 48 response level estimates in table 6, 
the estimates for 34 (48-14) response levels are 
included in the summaries. 



Figure 3 presents the reduction in bias estimated 
using scheme Al for the SR characteristics, and 
figure 4 is the same representation for the SS&D 
items. The reduction in bias estimates presented 
are the absolute values of the bias estimates 
shown in tables 5 and 6 after deleting the 
estimates for one response level per item. For 
both components, the bias reductions are small. 
The largest absolute bias is 1.3 percent for SR 
and 0.9 percent for SS&D. The mean and 
median of the bias reductions and the absolute 
values of the bias reductions were also computed 
for each scheme and each component. For the 
SR component, the mean and median of the 
absolute value of the estimated bias reductions 
are between 0.2 and 0.4 percent. For the SS&D, 
the mean and median of the absolute values are 
between 0.1 and 0.3. 



Table 5.— Estimated reduction in bias and bias ratio for selected characteristics of the School Readiness 
component 





Standard estimate 


Estimated reduction in bias 1 


Bias ratio 


Characteristic 


Percent 


Standard 

error 


Scheme 

Al 


Scheme 

A2 


Scheme 

B1 


Scheme 

B2 


Scheme 

Al 


Scheme 

A2 


Scheme 

B1 


Scheme 

B2 


Parental educational level 
Less than high school graduate 


8.6 


0.3 


-1.7 


-1.9 


0.1 


0.1 


-5.7 


-6.3 


0.3 


0.3 


High school graduate or equivalent 


33.9 


0.8 


0.4 


0.3 


-0.7 


-1.0 


0.5 


0.4 


-0.9 


-1.3 


Some college 


57.5 


0.7 


1.3 


1.6 


0.6 


0.9 


1.9 


2.3 


0.9 


1.3 


Mother’s employment status 
No mother in household 


2.4 


0.2 


-0.1 


-0.1 


-0.1 


-0.1 


-0.5 


-0.5 


-0.5 


-0.5 


Employed 35 hours/week or more 


34.3 


0.5 


0.5 


0.8 


0.2 


0.5 


1.0 


1.6 


0.4 


1.0 


Employed less than 35 hours/week 


20.9 


0.5 


- 0.1 


-0.2 


0.0 


-0.2 


-0.2 


-0.4 


0.0 


-0.4 


Seeking employment 


6.6 


0.4 


0.0 


-0.1 


-0.1 


-0.1 


0.0 


-0.3 


-0.3 


-0.3 


Not in labor force 


35.8 


0.6 


-0.4 


-0.3 


0.0 


0.0 


-0.7 


-0.5 


0.0 


0.0 


Father’s employment status 
No father in household 


26.3 


0.5 


-0.4 


-0.6 


0.0 


-0.1 


-0.8 


-1.2 


0.0 


-0.2 


Employed 35 hours/week or more 


63.4 


0.6 


0.3 


0.5 


0.1 


0.2 


0.5 


0.8 


0.2 


0.3 


Employed less than 35 hours/week 


3.8 


0.3 


0.0 


-0.1 


0.0 


0.1 


0.0 


-0.3 


0.0 


0.3 


Seeking employment 


3.2 


0.3 


0.0 


0.0 


-0.1 


-0.2 


0.0 


0.0 


-0.3 


-0.7 


Not in labor force 


3.3 


0.2 


0.1 


0.2 


0.0 


0.1 


0.5 


1.0 


0.0 


0.5 


Median household income in ZIP Code 
$15,000 or less 


4.0 


0.4 


-0.2 


0.0 


0.1 


0.2 


-0.5 


0.0 


0.3 . 


0.5 


$15,000 to $25,000 


30.1 


0.8 


-0.6 


-0.5 


-0.3 


-0.5 


-0.7 


-0.6 


-0.4 


-0.6 


More than $25,000 


66.0 


0.8 


0.8 


0.6 


0.4 


0.4 


1.0 


.0.7 


0.5 


0.5 


Mobility in ZIP Code 

High 


7.6 


0.5 


0.0 


0.2 


0.0 


0.2 


0.0 


0.4 


0.0 


0.4 


Medium/high 


17.7 


0.6 


-0.1 


-0.2 


-0.3 


-0.3 


-0.2 


-0.3 


-0.5 


-0.5 


Medium 


30.0 


0.7 


0.0 


-0.1 


0.0 


0.0 


0.0 


-0.1 


0.0 


0.0 


Medium/low 


31.2 


0.8 


0.3 


0.3 


0.4 


0.2 


0.4 


0.4 


0.5 


0.2 


Low 


13.5 


0.5 


-0.2 


-0.2 


-0.1 


0.0 


-0.4 


-0.4 


-0.2 


0.0 


Time since doctor visit for routine care 
Less than 1 year 


84.1 


0.4 


0.4 


0.4 


0.2 


0.1 


1.0 


1.0 


0.5 


0.2 


Over 1 year ; 


15.9 


0.4 


-0.4 


-0.5 


-0.2 


-0.1 


-1.0 


-1.3 


-0.5 


-0.2 
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Table 5.~ Estimated reduction in bias and bias ratio for selected characteristics of the School Readiness 
component (continued) 





Standard estimate 


Estimated reduction in bias 1 




Bias ratio 2 




Characteristic 


Percent 


Standard 

error 


Scheme 

A1 


Scheme 

A2 


Scheme 

B1 


Scheme 

B2 


Scheme 

A1 


Scheme 

A2 


Scheme 

B1 


Scheme 

B2 


Birth weight 

5.5 pounds or less 


93.3 


0.3 


-0.1 


0.0 


0.0 


0.1 


-0.3 


0.0 


0.0 


0.3 


Greater than 5.5 pounds 


6.7 


0.3 


0.1 


0.0 


0.0 


-0.1 


0.3 


0.0 


0.0 


-0.3 


Child attending center-based program* 






















Yes 


52.6 


0.8 


0.9 


0.3 


0.8 


0.6 


1.1 


0.4 


1.0 


0.8 


No 


47.4 


0.8 


-0.9 


-0.3 


-0.8 


-0.6 


-1.1 


-0.4 


-1.0 


-0.8 


Child ever attended center-based program 3 






















Yes 


62.9 


0.8 


0.5 


0.3 


0.4 


0.3 


0.6 


0.4 


0.5 


0.4 


No 


37.1 


0.8 


-0.5 


-0.3 


-0.4 


-0.3 


-0.6 


-0.4 


.-0.5 


-0.4 


Attended center-based program prior to 






















Yes 


73.5 


0.5 


0.6 


0.7 


0.5 


0.6 


1.2 


1.4 


1.0 


1.2 


No 


26.5 


0.5 


-0.6 


-0.7 


-0.5 


-0.6 


-1.2 


-1.4 


-1.0 


-1.2 


Women, Infant, and Children program 
participant 3 






















Yes 


33.8 


1.0 


-0.6 


-0.1 


-0.8 


-0.7 


-0.6 


-0.1 


-0.8 


-0.7 


No 


66.2 


1.0 


0.6 


0.1 


0.8 


0.7 


0.6 


0.1 


0.8 


0.7 


Free meal at school or center 4 






















Yes 


35.8 


0.6 


-0.9 


-1.1 


-0.5 


-0.5 


-1.5 


-1.8 


-0.8 


-0.8 


No 


64.2 


0.6 


0.9 


1.1 


0.5 


0.5 


1.5 


1.8 


0.8 


0.8 


Repeated kindergarten 5 
Yes 


5.7 


0.4 


-0.3 


-0.5 


-0.2 


-0.2 


-0.8 


-1.3 


-0.5 


-0.5 


No 


94.3 


0.4 


0.3 


0.5 


0.2 


0.2 


0.7 


1.3 


0.5 


0.5 


Family member read to child in last week 6 
Not in last week 


4.3 


0.5 


-0.3 


-0.4 


0.1 


0.1 


-0.6 


-0.8 


0.2 


0.2 


Once or twice 


16.9 


0.9 


0.0 


0.4 


0.0 


0.2 


0.0 


0.4 


0.0 


0.2 


Three or more times 


78.8 


0.9 


0.3 


0.0 


-0.1 


-0.3 


0.3 


0.0 


-0.1 


-0.3 


Taught child letters or words in last week 6 






















Not in last week 


12.3 


0.4 


-0.4 


-0.4 


-0.3 


-0.3 


-1.0 


-1.0 


-0.7 


-0.7 


Once or twice 


27.4 


0.6 


0.3 


0.3 


0.3 


0.5 


0.5 


0.5 


0.5 


0.8 


Three or more times 


60.3 


0.7 


0.1 


0.1 


0.0 


-0.2 


0.1 


0.1 


0.0 


-0.3 


Taught songs or music in last week 6 






















Not in last week 


33.3 


0.6 


0.6 


0.2 


0.3 


-0.1 


1.0 . 


0.3 


0.5 


-0.2 


Once or twice 


30.1 


0.7 


-0.3 


-0.1 


-0.2 


0.0 


-0.4 


-0.1 


-0.3 


0.0 


Three or more times 


36.7 


0.7 


-0.2 


0.1 


0.0 


0.1 


-0.3 


0.1 


0.0 


0.1 


Did arts or crafts with child in last week 6 






















Not in last week 


32.2 


0.7 


-0.3 


-0.7 


-0.5 


-1.0 


-0.4 


-1.0 


-0.7 


-1.4 


Once or twice 


35.7 


0.7 


0.1 


0.3 


-0.1 


0.0 


0.1 


0.4 


-0.1 


0.0 


Three or more times 


32.1 


0.6 


0.2 


0.4 


0.6 


1.0 


0.3 


0.7 


1.0 


1.7 


Visited library with child in last month 6 






















Yes 


39.4 


0.9 


0.5 


0.4 


0.4 


0.5 


0.6 


0.4 


0.4 


0.6 


No 


60.6 


0.9 


-0.5 


-0.4 


-0.4 


-0.5 


-0.6 


-0.4 


-0.4 


-0.6 


Visited zoo with child in last month 6 






















Yes 


16.8 


0.7 


0.3 


0.1 


0.2 


0.1 


0.4 


0.1 


0.3 


0.1 


No 


83.2 


0.7 


-0.3 


-0.1 


-0.2 


-0.1 


-0.4 


-0.1 


-0.3 


-0.1 



‘The estimated reduction in bias is the standard estimate of the percent minus the adjusted estimate of the percent. 

2 The bias ratio is the bias reduction estimate divided by the standard error. 

3 Estimate restricted to preschoolers. 

^Estimate applies to all children except preschoolers. 

’Estimate restricted to children in primary school. 

'’Estimate applies to all children except those in primary school. 

NOTE: Percents may not add to 100 because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Survey, spring 1993. 
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Table 6.~ Estimated reduction in bias and bias ratio for selected characteristics of the School Safety and 
Discipline component 





Standard estimate 


Estimated reduction in bias 1 




Bias ratio 2 




Characteristic 


Percent 


Standard 

error 


Scheme 

Al 


Scheme 

A2 


Scheme 

B1 


Scheme 

B2 


Scheme 

Al 


Scheme 

A2 


Scheme 

B1 


Scheme 

B2 


Parental educational level 
Less than high school graduate 


9.4 


0.5 


-1.2 


-1.3 


-0.3 


-0.6 


-2.4 


-2.6 


-0.6 


-1.2 


High school graduate or equivalent 


32.7 


0.6 


0.3 


0.0 


-0.2 


-0.6 


0.5 


0.0 


-0.3 


-1.0 


Some college 


57.9 


0.5 


0.9 


1.3 


0.5 


1.1 


1.8 


2.6 


1.0 


2.2 


Mothers employment status 






















No mother in household 


3.5 


0.2 


0.0 


0.0 


0.0 


0.0 


0.0 


0.0 


0.0 


0.0 


Employed 35 hours/week or more 


46.2 


0.5 


0.0 


0.1 


-0.1 


0.1 


0.0 


0.2 


-0.2 


0.2 


Employed less than 35 hours/week 


20.3 


0.5 


0.1 


0.0 


0.0 


-0.1 


0.2 


0.0 


0.0 


-0.2 


Seeking employment 


4.5 


0.3 


-0.2 


-0.2 


-0.2 


-0.2 


-0.7 


-0.7 


-0.7 


-0.7 


Not in labor force 


25.5 


0.5 


0.0 


0.1 


0.2 


0.2 


0.0 


0.2 


0.4 


0.4 


Father’s employment status 






















No father in household 


26.8 


0.6 


-0.2 


-0.2 


-0.1 


-0.2 


-0.3 


-0.3 


-0.2 


-0.3 


Employed 35 hours/week or more 


63.2 


0.5 


0.6 


0.9 


0.6 


0.8 


1.2 


1.8 


1.2 


1.6 


Employed less than 35 hours/ week 


3.1 


0.2 


-0.2 


-0.2 


-0.2 


-0.2 


-1.0 


-1.0 


-1.0 


-1.0 


Seeking employment 


2.6 


0.2 


-0.2 


-0.3 


-0.2 


-0.3 


-1.0 


-1.5 


-1.0 


-1.5 


Not in labor force 


4.3 


0.3 


-0.1 


-0.1 


-0.1 


-0.1 


-0.3 


-0.3 


-0.3 


-0.3 


Median household income in ZIP Code 






















$15,000 or less 


4.2 


0.3 


-0.1 


-0.2 


0.0 


-0.1 


-0.3 


-0.7 


0.0 


-0.3 


$15,000 to $25, 000 


31.3 


0.7 


-0.3 


-0.4 


-0.3 


-0.4 


-0.4 


-0.6 


-0.4 


-0.6 


More than $25,000 


64.5 


0.8 


0.5 


0.6 


0.3 


0.5 


0.6 


0.8 


0.4 


0.6 


Mobility in ZIP Code 






















High 


5.5 


0.3 


0.2 


0.1 


0.1 


0.1 


0.7 


0.3 


0.3 


0.3 


Medium/high 


17.1 


0.6 


0.0 


0.1 


0.0 


0.0 


0.0 


0.2 


0.0 


0.0 


Medium 


29.4 


0.6 


-0.1 


-0.1 


-0.1 


-0.2 


-0.2 


-0.2 


-0.2 


-0.3 


Medium/low 


33.9 


0.7 


-0.1 


-0.1 


0.0 


-0.1 


-0.1 


-0.1 


0.0 


-0.1 


Low 


14.1 


0.7 


0.0 


0.1 


0.1 


0.1 


0.0 


0.1 


0.1 


0.1 


School control 






















Public 


91.2 


0.3 


-0.1 


-0.1 


-0.1 


-0.1 


-0.3 


-0.3 


-0.3 


-0.3 


Private 


8.8 


0.3 


0.1 


0.1 


0.1 


0.1 


0.3 


0.3 


0.3 


0.3 


Visitors required to sign in at school 






















Yes 


79.9 


0.5 


0.1 


0.4 


0.0 


0.2 


0.2 


0.8 


0.0 


0.4 


No 


20.1 


0.5 


-0.1 


-0.4 


0.0 


-0.2 


-0.2 


-0.8 


0.0 


-0.4 


Students in fighting gangs at school 3 






















Yes 


22.3 


0.5 


-0.3 


-0.4 


-0.3 


-0.5 


-0.6 


-0.8 


-0.6 


-1.0 


No 


77.7 


0.5 


0.3 


0.4 


0.3 


0.5 


0.6 


0.8 


0.6 


1.0 


Ease of obtaining marijuana at school 3 






















Very or fairly easy 


39.2 


0.6 


-0.2 


-0.3 


-0.2 


-0.3 


-0.3 


-0.5 


-0.3 


-0.5 


Hard 


29.7 


0.5 


0.1 


0.1 


0.2 


0.2 


0.2 


0.2 


0.4 


0.4 


Nearly impossible 


31.1 


0.6 


0.1 


0.1 


0.0 


0.1 


0.2 


0.2 


0.0 


0.2 


Had drug or alcohol ed program this year 






















Yes 


68.5 


0.7 


0.6 


0.8 


0.7 


0.9 


0.9 


1.1 


1.0 


1.3 


No 


31.5 


0.7 


-0.6 


-0.8 


-0.7 


-0.9 


-0.9 


-1.1 


-1.0 


-1.3 


Fear of incident of crime at school 






















None 


66.1 


0.5 


0.0 


0.0 


0.0 


0.0 


0.0 


0.0 


0.0 


0.0 


Fear of theft or robbery 4 


11.9 


0.5 


-0.1 


-0.2 


0.0 


-0.2 


-0.2 


-0.4 


0.0 


-0.4 


Fear of bullying or assault 4 


8.6 


0.3 


-0.1 


-0.1 


-0.1 


-0.1 


-0.3 


-0.3 


-0.3 


-0.3 


Fear of two or more types of incidents 4 


13.3 


0.5 


0.1 


0.3 


0.1 


0.2 


0.2 


0.6 


0.2 


0.4 


Knowledge of crime at school 






















None 


38.7 


0.6 


0.2 


0.1 


0.2 


0.1 


0.3 


0.2 


0.3 


0.2 


Fear of theft or robbery 4 


14.1 


0.5 


0.2 


0.3 


0.2 


0.3 


0.4 


0.6 


0.4 


0.6 


Fear of bullying or assault 4 


15.6 


0.4 


-0.5 


-0.4 


-0.4 


-0.4 


-1.3 


-1.0 


-1.0 


-1.0 


Fear of two or more types of incidents 4 


31.6 


0.6 


0.1 


0.0 


0.0 


0.0 


0.2 


0.0 


0.0 


0.0 


Victimization by crime 






















Not victimized 


73.0 


0.5 


0.3 


0.2 


0.3 


0.2 


0.6 


0.4 


0.6 


0.4 


Victim of theft or robbery 4 


10.9 


0.3 


-0.2 


-0.1 


-0.1 


0.0 


-0.7 


-0.3 


-0.3 


0.0 


Victim of bulling or assault 4 


8.9 


0.3 


-0.1 


0.0 


-0.2 


-0.1 


-0.3 


0.0 


-0.7 


-0.3 


Victim of two or more types of incidents 4 


7.2 


0.3 


0.0 


0.0 


0.0 


-0.1 


0.0 


0.0 


0.0 


-0.3 
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Table 6.-- Estimated reduction in bias and bias ratio for selected characteristics of the School Safety and 
Discipline component (continued) 





Standard estimate 


Estimated reduction in bias 1 




Bias ratio 2 




Characteristic 


Percent 


Standard 

error 


Scheme 

Al 


Scheme 

A2 


Scheme 

B1 


Scheme 

B2 


Scheme 

Al 


Scheme 

A2 


Scheme 

B1 


Scheme 

B2 


Witnessed crime at school 
None 


63.8 


0.8 


0.2 


0.2 


0.2 


0.2 


0.2 


0.2 


0.2 


0.2 


Witnessed robbery 5 


0.6 


0.1 


0.0 


0.0 


0.0 


0.0 


0.0 


0.0 


0.0 


0.0 


Witnessed bulling or assault 5 


24.1 


0.8 


-0.3 


-0.3 


-0.3 


-0.3 


-0.4 


-0.4 


-0.4 


-0.4 


Witnessed two or more types of incidents 


11.4 


0.4 


0.0 


0.1 


0.0 


0.0 


0.0 


0.2 


0.0 


0.0 



‘The estimated reduction in bias is the standard estimate of the percent minus the adjusted estimate of the percent. 

2 The bias ratio is the bias reduction estimate divided by the standard error. 

3 Item was only asked for students in grades 6 through 1 2. 

4 For the fear of incident, knowledge of crime, and victimized by crime variables, the second response category is used if either theft or robbery was reported but not both, the 
third response category is used if either bullying or assault was reported but not both. 

5 This response category is used if either bullying or assault was reported, but not both. 

NOTE: Percents may not add to 100 because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Survey, spring 1993. 



Figure 3.— Estimated reduction in absolute bias for School Readiness characteristics (scheme Al) 




Percent of children 



SOURCE: U.S . Department of Education, National Center for Education Statistics, National Household Education Survey, spring 1 993 . 
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Figure 4.-- Estimated reduction in absolute value of bias for School Safety and Discipline 
characteristics (scheme Al) 
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SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Survey, spring 1993. 



Bias Ratio 

The size of the absolute reduction in bias is not a 
very useful statistical measure of the impact of 
the bias because it does not take the magnitude of 
the standard error of the estimate into account. 
Cochran (1977) discusses the impact on 
confidence intervals as the ratio of the bias to the 
sampling error varies. The bias ratio is defined 
as the bias divided by the sampling error of the 
estimate. For each scheme the bias ratio is given 
by 



r n =' 



se(p s ) 



(4) 



with the sampling error of the standard estimate 
as the denominator. As the bias ratio increases, 
the chance of covering the population value 
departs significantly from the nominal 
confidence interval. For example, a bias ratio of 
0.1 has very little impact on a 95 percent 
confidence interval, but a bias ratio of 1.0 results 
in a nominal 95 percent confidence interval that 
only covers the population value 83 percent of 
the time. 



The bias ratios for all of the response categories 
for the SR items are given in table 5, and the 
ratios for the SS&D items are in table 6. The 
discussion of the bias ratios that follows is based 
on the ratios remaining after eliminating one 
response level for each item. 

Many of the bias ratios for the SR items are 
large, even though the average and median ratios 
are near zero. Nearly half of the ratios are larger 
than 0.4 in absolute value. A ratio of 0.4 is large 
enough to reduce a nominal confidence interval 
from 95 percent to about 93 percent. For the 
SS&D items, the bias ratios are smaller. Only 5 
of the 34 bias ratio estimates in table 6 are 
greater than 0.4. 

The bias ratios show that the biases as estimated 
under the assumed model could have an effect on 
the inferences made from the survey estimates. 
The effect on the inferences is a greater problem 
for the SR component than for the SS&D 
component. The confidence intervals based on 
the standard estimates for some characteristics 
will not attain the nominal confidence intervals 
due to the undercoverage bias. 
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Consistency Across Adjustment Schemes 

A review of the estimates in tables 5 and 6 shows 
that if the bias reduction estimated under one 
adjustment scheme is large and positive, the 
estimates under the other schemes tend to be 
large and positive. The correlations between the 
bias estimates under the four schemes are a 
measure of the consistency of the reduction in 
bias estimates across the schemes. In general, 
the correlations 6 are very high, as might be 
expected. For the SR component, the correlation 
between the estimates goes from a low of 0.6 
between the estimates for schemes A2 and B 1 , to 
a high of 0.9 for schemes A1 and A2 and 
schemes B1 and B2. The correlations for the 
bias reduction estimates from the characteristics 
of the SS&D component are uniformly high, with 
correlations nearly 0.9 between all the schemes. 

The bias adjustments resulting from defining the 
cells by educational attainment within 
race/ethnicity are highly correlated with those 
formed by tenure within race/ethnicity. 
However, the consistency of the bias estimates 
does not imply that all of the adjustment schemes 
are equivalent in terms of their overall statistical 
properties. So far, the discussion has only been 
about the bias reduction. Their variance 
implications are presented below. 

Variance Implications 

The results above show that the standard 
estimates from the NHES:93 are subject to 



5 The correlations were computed after deleting one of the 
response categories for each item, as discussed above. 



coverage bias under the assumed model. Since 
the ordinary measure of variation for an unbiased 
estimate, the variance, is not appropriate for 
biased estimates, the mean square errors of the 
biased estimates are considered. The mean 
square error of the estimate (MSE) is a 
frequently used statistic that reflects both the 
variation about the average and the bias of an 
estimate. The MSE is the sum of the variance 
and the square of the bias of the estimate. 

The MSE of the standard estimate can be 
computed by using the standard variance and 
bias estimates presented above. 7 The estimated 
MSE can be written as 

mse fl = vai(p s ) + bg (5) 

where the terms have been defined before. Four 
different estimates of the mean square error can 
be formed, one corresponding to each of the 
weighting schemes. Only the estimated mean 
square errors for scheme A1 are shown in tables 
7 and 8 because the results for the other schemes 
are so similar. 

It is interesting to note that the mean square error 
of the estimate is functionally related to the bias 
ratio discussed previously. The relationship for 
the standard NHES:93 estimate is given by 

mse a = var( p s )(1 + r fl 2 ) . (6) 

This relationship explains why the value of the 
mean square error is close to the variance 
estimate unless the bias ratio is large. 



6 An unbiased estimate of the MSE can be found by adding an 

unbiased estimate of the variance to an unbiased estimate of 
the bias squared. The estimated bias squared is not 
technically an unbiased estimate of the squared bias, but the 
difference is extremely small in this case. Consequently, the 
squared bias estimates are used for this report. 
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Table 1 .— Estimated mean square error and mean square ratio for selected characteristics of the School 
Readiness component 





Standard estimate 




Mean square ratio 2 




Characteristic 


Percent 


Variance 


MSE 1 


Scheme 

A1 


Scheme 

A2 


Scheme 

B1 


Scheme 

B2 


Parental educational level 












3.5 


3.8 


Less than high school graduate 


8.6 


0.1 


3.0 


3.6 


4.1 


High school graduate or equivalent 


33.9 


0.6 


0.8 


96.0 


108.0 


92.8 


100.8 


Some college 


57.5 


0.5 


2.2 


27.0 


30.3 


26.1 


28.3 


Mother's employment status 










108.0 


92.8 


100.8 


No mother in household 


2.4 


0.0 


0.1 


96.0 


Employed 35 hours/week or more 


34.3 


0.3 


0.5 


60.0 


67.5 


58.0 


63.0 


Employed less than 35 hours/ week 


20.9 


0.3 


0.3 


115.4 


129.8 


111.5 


121.2 


Seeking employment 


6.6 


0.2 


0.2 


120.0 


135.0 


116.0 


126.0 


Not in labor force 


35.8 


0.4 


0.5 


83.1 


93.5 


80.3 


87.2 


Father's employment status 
No father in household 


26.3 


0.3 


0.4 


73.2 


82.3 


70.7 


76.8 


Employed 35 hours/week or more 


63.4 


0.4 


0.4 


96.0 


108.0 


92.8 


100.8 


Employed less than 35 hours/week 


3.8 


0.1 


0.1 


120.0 


135.0 


116.0 


126.0 


Seeking employment 


3.2 


0.1 


0.1 


120.0 


135.0 


116.0 


126.0 


Not in labor force 


3.3 


0.0 


0.0 


96.0 


108.0 


92.8 


100.8 


Median household income in ZIP Code 
















$15,000 or less 


4.0 


0.2 


0.2 


96.0 


108.0 


92.8 


100.8 


$15,000 to $25,000 


30.1 


0.6 


1.0 


76.8 


86.4 


74.2 


80.6 


More than $25,000 


66.0 


0.6 


1.3 


60.0 


67.5 


58.0 


63.0 


Mobility in ZIP Code 














126.0 


High 


7.6 


0.3 


0.3 


120.0 


135.0 


116.0 


Medium/high 


17.7 


0.4 


0.4 


116.8 


131.4 


112.9 


122.6 


Medium 


30.0 


0.5 


0.5 


120.0 


135.0 


116.0 


126.0 


Medium/low 


31.2 


0.6 


0.7 


105.2 


118.4 


101.7 


110.5 


Low 


13.5 


0.3 


0.3 


103.4 


116.4 


100.0 


108.6 


Time since doctor visit for routine care 












58.0 


63.0 


Less than 1 year 


84.1 


0.2 


0.3 


60.0 


67.5 


Over 1 year 


15.9 


0.2 


0.3 


60.0 


67.5 


58.0 


63.0 


Birth weight 














113.4 


5.5 pounds or less 


93.3 


0.1 


0.1 


108.0 


121.5 


104.4 


Greater than 5.5 pounds 


6.7 


0.1 


0.1 


108.0 


121.5 


104.4 


113.4 


Child attending center-based program 2 








53.0 


59.6 


51.2 


55.6 


Yes 


52.6 


0.6 


1.5 


No 


47.4 


0.6 


1.5 


53.0 


59.6 


51.2 


55.6 


Child ever attended center-based program 3 
















Yes 


62.9 


0.6 


0.9 


86.3 


97.1 


83.4 


90.6 


No 


37.1 


0.6 


0.9 


86.3 


97.1 


83.4 


90.6 


Attended center-based program prior to school 4 














51.6 


Yes . 


73.5 


0.3 


0.6 


49.2 


55.3 


47.5 


No 


26.5 


0.3 


0.6 


49.2 


55.3 


47.5 


51.6 


Women, Infants, and Children program participant 2 














92.6 


Yes 


33.8 


1.0 


1.4 


88.2 


99.3 


85.3 


No 


66.2 


1.0 


1.4 


88.2 


99.3 


85.3 


92.6 


Free meal at school or center A 
















Yes 


35.8 


0.4 


1.2 


36.9 


41.5 


35.7 


38.8 


No : 


64.2 


0.4 


1.2 


36.9 


41.5 


35.7 


38.8 


Repeated kindergarten 5 










86.4 


74.2 


80.6 


Yes 


5.7 


0.2 


0.3 


76.8 


No 


94.3 


0.2 


0.2 


76.8 


86.4 


74.2 


80.6 


Family member read to child in last week 6 
Not in last week 


4.3 


0.3 


0.3 


88.2 


99.3 


85.3 


92.6 


Once or twice 


16.9 


0.8 


0.8 


120.0 


135.0 


116.0 


126.0 


Three or more times 


78.8 


0.8 


0.9 


108.0 


121.5 


104.4 


113.4 


Taught child letters or words in last week 6 
Not in last week 


12.3 


0.2 


0.3 


60.0 


67.5 


58.0 


63.0 


Once or twice 


27.4 


0.4 


0.4 


96.0 


108.0 


92.8 


100.8 


Three or more times 


60.3 


0.5 


0.5 


117.6 


132.3 


113.7 


123.5 
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Table 7.- Estimated mean square error and mean square ratio for selected characteristics of the School 
Readiness component (continued) 





Standard estimate 




Mean square ratio 2 




Characteristic 


Percent 


Variance 


MSE' 


Scheme 

Al 


Scheme 

A2 


Scheme 

B1 


Scheme 

B2 


Taught songs or music in last week 6 
















Not in last week 


33.3 


0.4 


0.7 


60.0 


67.5 


58.0 


63.0 


Once or twice 


30.1 


0.5 


0.6 


101.4 


114.1 


98.0 


106.4 


Three or more times 


36.7 


0.5 


0.5 


110.9 


124.8 


107.2 


116.5 


Did arts or crafts with child in last week 6 
















Not in last week 


32.2 


0.5 


0.6 


101.4 


114.1 


98.0 


106.4 


Once or twice 


35.7 


0.5 


0.5 


117.6 


132.3 


113.7 


123.5 


Three or more times 


32.1 


0.4 


0.4 


108.0 


121.5 


104.4 


113.4 


Visited library with child in last month 6 
















Yes 


39.4 


0.8 


1.1 


91.7 


103.2 


88.6 


96.3 


No 


60.6 


0.8 


1.1 


91.7 


103.2 


88.6 


96.3 


Visited zoo with child in last month 6 
















Yes 


16.8 


0.5 


0.6 


101.4 


114.1 


98.0 


106.4 


No 


83.2 


0.5 


0.6 


101.4 


114.1 


98.0 


106.4 



! MSE is the estimated variance phis the square of the bias estimate using scheme Al. 

2 The mean square ratio is the mean square error erf the adjusted estimate divided by die mean square error of the standard estimate. 
^Estimate restricted to preschoolers. 

4 Estimate applies to all children except preschoolers. 

5 Estimate restricted to children in primary school. 

^Estimate applies to all children except those in primary school 
NOTE: Percents may not add to 100 because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Survey, spring 1993. 
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Table 8.-- Estimated mean square error and mean square ratio for selected characteristics of the School 
Safety and Discipline component 





Standard estimate 




Mean square ratio 2 




Characteristic 


Percent 


Variance 


MSE 1 


Scheme 

A1 


Scheme 

A2 


Scheme 

B1 


Scheme 

B2 


Parental educational level 










18.6 


16.4 


. 18.5 


Less than high school graduate ; 


9.4 


0.3 


1.7 


16.6 


High school graduate or equivalent 


32.7 


0.4 


0.5 


89.6 


100.8 


88.8 


100.0 


Some college 


57.9 


0.3 


1.1 


26.4 


29.7 


26.2 


29.5 


Mother's employment status 










126.0 


111.0 


125.0 


No mother in household 


3.5 


0.0 


0.0 


112.0 


Employed 35 hours/week or more 


46.2 


0.3 


0.3 


112.0 


126.0 


111.0 


125.0 


Employed less than 35 hours/week 


20.3 


0.3 


0.3 


107.7 


121.2 


106.7 


120.2 


Seeking employment 


4.5 


0.1 


0.1 


77.5 


87.2 


76.8 


86.5 


Not in labor force 


25.5 


0.3 


0.3 


112.0 


126.0 


11 1.0 


125.0 


Father's employment status 










113.4 


99.9 


112.5 


No father in household 


26.8 


0.4 


0.4 


100.8 


Employed 35 hours/week or more 


63.2 


0.3 


0.6 


45.9 


51.6 


45.5 


51.2 


Employed less than 35 hours/week 


3.1 


0.0 


0.1 


56.0 


63.0 


55.5 


62.5 


Seeking employment 


2.6 


0.0 


0.1 


56.0 


63.0 


55.5 


62.5 


Not in labor force 


4.3 


0.1 


0.1 


100.8 


113.4 


99.9 


112.5 


Median household income in ZIP Code 














112.5 


$15,000 or less 


4.2 


0.1 


0.1 


100.8 


113.4 


99.9 


$15,000 to $25,000 


31.3 


0.5 


0.6 


94.6 


106.4 


93.8 


105.6 


More than $25,000 


64.5 


0.6 


0.9 


80.5 


90.6 


79.8 


89.9 


Mobility in ZIP Code 












76.8 


86.5 


High 

Medium/high 


5.5 


0.1 


0.1 


77.5 


87.2 


17.1 


0.4 


0.4 


112.0 


126.0 


111.0 


125.0 


Medium 


29.4 


0.4 


0.4 


109.0 


122.6 


108.0 


121.6 


Medium/low 


33.9 


0.5 


0.5 


109.8 


123.5 


108.8 


122.5 


Low 


14.1 


0.5 


0.5 


112.0 


126.0 


111.0 


125.0 


School control 














112.5 




91.2 


0.1 


0.1 


100.8 


113.4 


99.9 


Private 


8.8 


0.1 


0.1 


100.8 


113.4 


99.9 


112.5 


Visitors required to sign in at school 










121.2 


106.7 


120.2 


Yes ' 


79.9 


0.3 


0.3 


107.7 


No 


20.1 


0.3 


0.3 


107.7 


121.2 


106.7 


120.2 


Students in fighting gangs at school 3 








83.1 


93.4 


80.1 


91.2 


Yes 


22.3 


0.3 


0.3 


No 


77.7 


0.3 


0.3 


83.1 


93.4 


80.1 


91.2 


Ease of obtaining marijuana at school 3 








101.7 


114.3 


98.1 


111.6 


Very or fairly easy 


39.2 


0.4 


0.4 


Hard 


29.7 


0.3 


0.3 


108.7 


122.1 


104.8 


119.2 


Nearly impossible 


31.1 


0.4 


0.4 


109.9 


123.6 


106.1 


120.6 


Had drug or alcohol ed program this year 








64.6 


72.6 


64.0 


72.1 


Yes 


68.5 


0.5 


0.8 


No 


31.5 


0.5 


0.9 


64.6 


72.6 


64.0 


72.1 


Fear of incident of crime at school 












111.0 


125.0 


None 


66.1 


0.3 


0.3 


112.0 


126.0 


Fear of theft or robbery 4 


11.9 


0.3 


0.3 


107.7 


121.2 


106.7 


120.2 


Fear of bullying or assault 4 


8.6 


0.1 


0.1 


100.8 


113.4 


99.9 


112.5 


Fear of two or more types of incidents 4 


13.3 


0.3 


0.3 


107.7 


121.2 


106.7 


120.2 


Knowledge of crime at school 








100.8 


113.4 


99.9 


112.5 


None 


38.7 


0.4 


0.4 


Fear of theft or robbery 4 


14.1 


0.3 


0.3 


96.6 


108.6 


95.7 


107.8 


Fear of bullying or assault 4 


15.6 


0.2 


0.4 


43.7 


49.2 


43.3 


48.8 


Fear of two or more types of incidents 4 


31.6 


0.4 


0.4 


109.0 


122.6 


108.0 


121.6 


Victimization by crime 












81.6 


91.9 


Not victimized , 


73.0 


0.3 


0.3 


82.4 


92.6 


Victim of theft or robbery 4 


10.9 


0,1 


0.1 


77.5 


87.2 


76.8 


86.5 


Victim of bulling or assault 4 


8.9 


0.1 


0.1 


100.8 


113.4 


99.9 


112.5 


Victim of two or more types of incidents 4 


7.2 


0.1 


0.1 


112.0 


126.0 


111.0 


125.0 
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Table 8.-- Estimated mean square error and mean square ratio for selected characteristics of the School 
Safety and Discipline component (continued) 





Standard estimate 


Mean square error ratio 


i 2 


Characteristic 


Percent 


Variance 


MSE' 


Scheme 

A1 


Scheme 

A2 


Scheme 

B1 


Scheme 

B2 


Witnessed crime at school 

None 


63.8 


0.6 


0.7 


105.4 


118.6 


104.5 


117.6 


Witnessed robbery 


0.6 


0.0 


0.0 


112.0 


126.0 


111.0 


125.0 


Witnessed bulling or assault 5 


24.1 


0.6 


0.7 


98.2 


110.5 


97.3 


109.6 


Witnessed two or more types of incidents 


11.4 


0.2 


0.2 


112.0 


126.0 


111.0 


125.0 



‘MSE is the estimated variance plus the square of the bias estimate using scheme A1 . 



2 The mean square ratio is the mean square error of the adjusted estimate divided by the mean square err or of the standard estimate. 

3 Asked only for students in grades 6 through 1 2. 

For the fear of incident, knowledge of crime, and victimized by crime variables, the second response category is used if either theft or robbery was reported but not both, the 
third response category is used if either bullying or assault was reported but not both. 

^This response category is used if either bullying or assault was reported, but not both. 

NOTE: Percents may not add to 100 because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Survey, spring 1993. 



The estimated mean square errors of the 
estimates can be used to contrast the bias and 
variance in the standard estimate with the 
variance in the adjusted estimates. As before, the 
telephone service adjustments are assumed to 
reduce the coverage bias. If bias were the only 
factor to be considered, then the adjusted 
estimates would clearly be preferred and the only 
decision would be which of the four adjustments 
should be implemented. However, the 
adjustments increase the variability of the 
estimates. The trade-off between the bias 
reduction and the variance increase associated 
with the adjustments is discussed below. 

The size of the variance increase from adjusting 
the weights using the telephone service 
interruption data was expressed earlier as the 
variance inflation factor ( V1F ). The relative 
VI Fs for each of the four schemes are given in 
table 4. Multiplying the standard variance 
estimates by the relative VI F for the appropriate 
adjustment factor yields an approximate variance 
for the adjusted estimates. The variance 
estimates for the standard estimates are shown in 



the second column in tables 7 and 8. Multiplying 
these estimates by the appropriate ratios for the 
adjustment scheme give the approximate 
variances of the estimates for each adjustment 
scheme. 

To aid in comparing the weighting procedures, 
the ratio of the estimated variance of the adjusted 
estimate to the estimated mean square error for 
the standard estimate was tabulated. This 
estimate is called the mean square ratio. It can 
be expressed as 

100 x relativeVIF a x var( p .) 

™sr a (p) = -r- — . (7) 

mse M (p) 

Note that the mean square error is derived using 
the bias estimated from scheme A1 only, but it is 
used to compute the mean square ratios for all 
four schemes. This is done to make the ratios 
comparable across the schemes. The estimates 
were also computed using scheme B2 and the 
results are not sensitive to the scheme used for 
the bias. 
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The mean square ratios are useful in assessing 
the effectiveness of the adjustments since they 
include contributions from both the reduction in 
bias (in the mean square error estimates) and the 
variance (in the VIF). When the mean square 
ratio is 100, the variance of the adjusted estimate 
is exactly equal to the mean square error of the 
biased, standard estimate. This is the break-even 
point. A ratio less than 100 indicates that the 
bias reduction of the adjustment is greater than 
the variance increase so the adjusted estimate has 
a smaller mean square error than the standard 
estimate. A mean square ratio over 100 means 
that the variance increase associated with the 
adjustment is greater than the bias reduction and 
the unadjusted estimator has the smaller mean 
square error. 

The mean square ratios for the selected items for 
the SR and SS&D components are given in tables 



7 and 8, respectively. These ratios are 
summarized below. As before, the estimate for 
one response category for each item was deleted 
before summarizing the estimates. To give a 
more complete representation of the 
distributions, figure 5 displays the mean square 
ratios for the SR items and figure 6 displays the 
mean square ratios for the SS&D items. For both 
of these figures, the horizontal axis is the 
estimated percentage of children in the category. 
The break-even line, when the mean square ratio 
is 100, is shown on the charts. 

The distributions of mean square ratios for both 
components are very similar. The medians for 
schemes A1 and B1 (those based on interruptions 
of 1 week or more) are near the break-even point 
of 100. The means for these schemes are close to 
90 and the figures confirm that this is due to the 
skewed distributions of the mean square ratios. 



Table 9.- Summaries of distribution of mean square ratios for selected characteristics of School Readiness 
and School Safety and Discipline components 







Scheme 






A1 


A2 


B1 


B2 


School Readiness 
mean 


89.8 


101.0 


86.8 


94.2 


median 


96.0 


108.0 


92.8 


100.8 


minimum 


27.0 


30.3 


26.1 


28.3 


maximum 


120.0 


135.0 


116.0 


126.0 


School Safety and Discipline 
mean 


93.3 


104.9 


92.2 


103.9 


median 


100.8 


113.4 


99.9 


112.5 


minimum 


26.4 


29.7 


26.2 


29.5 


maximum 


112.0 


126.0 


111.0 


125.0 



SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education 
Survey, spring 1993. 
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Figure 5. --Estimated mean square ratios for selected School Readiness items (scheme Al) 




SOURCE: US. Department of Education, National Center for Education Statistics, National Household Education Survey, spring 1993. 



Figure 6.-Estimated mean square ratios for selected School Safety and Discipline items (scheme Al) 




Percent of children 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Survey, spring 1993. 
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Figure 7.--Estimated mean square ratios for selected School Safety and Discipline items, schemes A1 
and A2 




Perpent of children 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Survey, spring 1993. 



Another important observation about the 
distributions of the mean square ratios for 
schemes A1 and B1 is the size of the ratios at the 
extremes of the distribution. The maximum 
mean square ratios for both components is 120, 
while some ratios are as small as 26. This means 
the maximum increase in the mean square error 
of the estimates is 20 percent, while the 
reductions in mean square error for other 
estimates are large. Thus, the penalty associated 
with adjusting even when the estimate is not 
biased is modest, but the benefits of adjusting 
when it is needed can be quite large. 

The distributions for the mean square ratios for 
schemes A1 and B1 are approximately equal, and 
the choice of which of these schemes should be 
used may be determined by nonstatistical issues, 
such as availability of data and the other types of 
adjustments required in the survey. The mean 
square ratios show that the adjusted weights 
reduce the mean square error for about half the 
estimates considered when compared with the 
standard weights. 

The distributions of the mean square ratios for 
schemes A2 and B2 (those based on interruptions 



of 1 month or more) have medians and means 
that are greater than 100. Essentially, these mean 
square ratios are shifted upward when compared 
with those of schemes A1 and Bl. This is clear 
from figure 7, which displays the ratios for 
schemes A1 and A2 for the SS&D component 
estimates. Because estimates with smaller mean 
square ratios have lower mean square errors, the 
upward shift indicates that adjusting using the 
shorter telephone interruption period of 1 week 
or more (schemes A1 and B 1) is preferable to the 
longer period (schemes A2 and B2). This result 
shows that in the trade-off between the variance 
inflation (the longer time period has a larger 
variance) and the bias reduction (the longer time 
period has smaller bias), the variance inflation 
has a bigger impact on the mean square error. 

Since the mean square ratios were computed 
using the estimated bias from scheme A1 (see 
equation 7), this might favor those schemes with 
shorter telephone interruption periods. As 
mentioned above, the mean square ratios were 
also computed using the bias estimates from 
scheme B2. The distributions of the mean square 
ratios using the scheme B2 bias estimates are 
very similar to those using the A1 bias estimates. 



For example, in the SR component the means of 
the distributions of mean square ratios using 
scheme B2 bias estimates are 95.7 for scheme 
Al, 107.7 for scheme A2, 92.5 for scheme Bl, 
and 100.5 for scheme B2. The adjustments using 
the shorter time periods (Al and Bl) still 
dominate those using the longer time periods (A2 
and B2). 

Conclusions 

In most surveys conducted solely by telephone, 
the potential bias introduced by excluding 
persons living in nontelephone households is a 
major concern. If the percentage of the target 
population living in nontelephone households is 
relatively large and the characteristics of those 
persons are different from those who live in 
telephone households, then the estimates may be 
susceptible to significant coverage bias. 

One method of addressing this problem without 
resorting to other modes of data collection is to 
adjust the estimates using data collected from the 
responding households with telephones. Since 
having a telephone in the household is not a 
static phenomenon, a reasonable response 
propensity model leads to adjustments based on 
data on interruptions in telephone service. In 
essence, households that have had interruptions 
in service are assumed to be similar to 
households without telephones at the time of the 
survey. Actually, the model only assumes that 
the relationship between persons living in 
households with interruptions in service and 
those in households without telephones is closer 
than between persons in all telephone households 
and those in households without telephones. The 
weights for persons in households reporting an 
interruption in telephone service are increased to 
adjust for those without telephones. 

In the NHES:93, households were asked about 
interruptions in telephone service during the past 
12 months. Estimates of the percentage of 
children living in households with any 



interruptions in telephone service were computed 
separately for the two NHES:93 components. 
The estimated percentage of children differed 
somewhat for the two populations, with 12 
percent of the younger children (the SR 
component) living in households with some 
interruption in service and 9 percent of the older 
children living in households with this 
characteristic. The response to the questions 
about interruptions in telephone service were 
then used to adjust the standard weights. Four 
alternative weights were created depending on 
the length of the telephone interruption (at least 1 
week or at least 1 month) and demographic 
characteristics of the household. The alternative 
weights were then used to produce estimates of 
the bias reduction in the standard estimates. 

The bias reduction estimates computed under the 
assumed model showed that the coverage 
adjustments for the SR component could have an 
important effect on the inferences from the 
survey estimates. The estimates of the ratio of 
the bias to the standard error of the estimate 
demonstrated that confidence intervals for some 
of the estimates based on the standard estimates 
were not likely to attain the nominal confidence 
intervals due to the undercoverage bias. On the 
other hand, many of the bias ratios were small 
and not substantively important. 

Although the adjustments reduced bias, they also 
increased the variability of the estimates. The 
trade-off between the bias reduction and the 
variance increase was examined by comparing 
the estimated mean square error of the standard 
estimates (which were assumed to be biased 
because of the coverage problem) to the variance 
estimates from the alternative weights (assumed 
to be unbiased). These ratios are referred to as 
mean square ratios. 

The alternative weighting schemes performed 
differently with respect to the mean square ratios, 
even though they were consistent in terms of bias 
reduction. The schemes based on interruptions 
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of telephone service of 1 week or more (schemes 
A1 and Bl) were better than the schemes based 
on interruptions of 1 month or more (A2 and 
B2). The bias adjustments resulting from using 
educational attainment by race/ethnicity 
categories (Al) were roughly equivalent to those 
using tenure by race/ethnicity (Bl). Either 
scheme Al or Bl could be used if the adjustment 
procedure were adopted. 

The distributions of the mean square ratios show 
that about half the estimates could be improved 
using the telephone service interruption 
adjustments. Furthermore, even for those 
estimates that were less accurate due to the 
variance increases associated with the 
differential weights, the magnitude of the 
increases were not large. In other words, the 
penalty for adjusting when it did not reduce the 
coverage bias was not very great. These findings 
suggest that the adjustments should be seriously 
considered. 

The size of the sample is another factor that 
should be considered when evaluating the use of 
the telephone service interruption adjustment. 
Bias ratios increase with the sample size because 
the bias is not affected while the sampling error 
of the estimate (the denominator of the bias ratio) 
decreases. Thus, the adjustments should be more 
beneficial in surveys with large sample sizes 
where the bias ratios might be expected to be 
large. 

While the results of this study suggest that the 
adjustments could be useful for many estimates 
from telephone surveys, other studies are needed 
before the adjustments are recommended. As 
discussed earlier, the estimates of the mean 
square errors in this study were based on the 
assumption that the adjusted estimates reduced 
the bias of the estimates. This model assumption 
could not be verified because of the lack of 
benchmark data for comparison. The assumed 
model may be beneficial to the adjusted 
estimates in the sense that it results in lower 



estimates of the mean square errors for the 
adjusted estimates. Thus, the findings of this 
study should be taken as an indication that 
adjustment using data on interruptions in 
telephone service is a feasible method that 
requires further study and evaluation. 

The questions about interruptions in telephone 
service were included in the NHES:95 to further 
evaluate this method of adjustment for coverage 
bias. The NHES:95 has a survey component on 
adult education so that data on service 
interruptions will be obtained for virtually all 
types of households rather than being restricted 
to households with children as was the case with 
the NHES:93. In addition, the questions recently 
were added to the National Health Interview 
Survey, a survey conducted by the Census 
Bureau for the National Center for Health 
Statistics. The findings from this survey should 
be even more useful in evaluating the method 
because the survey covers households without 
telephones by in-person interviews, eliminating 
the need for the critical model assumption used 
in this study. 

In summary, the findings of this study are: 

• The coverage bias associated with 
households without telephones could be 
important for some statistics, even after the 
ordinary poststratification adjustments; 

• Data collected on telephone interruptions can 
be used to reduce this bias by using a 
response probability type of adjustment; 

• The benefits of the bias reduction appear to 
be large enough to offset the variance 
increases due to increased variability in the 
weights in this study, although the results 
may differ for different size samples; and 

• The findings are tentative because they rely 
on a variety of assumptions, and some of 
them are favorable to the adjustment. 
Further research is needed before the 
procedure can be recommended. 
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